Background: Due to improvements in early detection, treatment, and supportive care, the number of colorectal cancer (CRC) survivors is increasing; therefore, careful attention should always be paid to the second primary cancer (SPC) in treating these CRC patients. The present study attempts to determine the correlation and clinical aspects of CRC to other cancers in patients suffering from SPC involving CRC. Methods: From January 2002 and June 2010, 1,679 cancer cases, CRC was accompanied by SPC in 89 patients (5.3%), including 16 (18%) synchronous and 73 (82%) metachronous SPC patients. These patients were subsequently classified into two groups: the first group had CRC diagnosed first as CRC first (CRCF); and the second group had another type of cancer diagnosed before the diagnosis of CRC as other cancer first (OCF). Of these 73 patients, 22 (30.1%) were in the group of CRCF, whereas 51 (69.9%) were in the group of OCF. Patients’ clinicopathological characteristics and clinical outcomes were analyzed and compared between the two groups. Results: There was a significant difference in the sites of cancers between the two groups: 14 (27.5%) patients in the OCF group had gastric cancer, compared to one (4.5%) patient in the CRCF group (P = 0.026). Although there was no difference of hepatitis B virus (HBV) or hepatitis C virus (HCV) carriers between the OCF and CRCF groups (P = 0.165), there were six (27.3%) CRC patients with hepatocellular carcinoma (HCC) in the CRCF group, which was significantly higher than the two (3.9%) patients in the OCF group (P = 0.003). Furthermore, the cancer-specific survival rate of the CRCF patient group was significantly higher than that of the OCF patient group (P = 0.036). Conclusions: In this retrospective analysis, gastric cancer patients compared to other secondary cancers were at a higher risk of developing subsequent CRC as SPC; alternatively, patients with CRC were at a higher risk of developing HCC as SPC subsequently, no matter whether patients were HBV or HCV carriers. Therefore, careful attention should always be paid to the possibility of secondary CRC to construct effective surveillance when treating cancer patients.